SNOW-2866776: add private var in numeric type#4022
Conversation
src/snowflake/snowpark/types.py
Outdated
| def __init__(self, **kwargs) -> None: | ||
| self._precision = kwargs.get("precision", None) | ||
| self._scale = kwargs.get("scale", None) |
There was a problem hiding this comment.
| def __init__(self, **kwargs) -> None: | |
| self._precision = kwargs.get("precision", None) | |
| self._scale = kwargs.get("scale", None) | |
| def __init__(self, precision: int | None = None, scale: int | None = None) -> None: | |
| self._precision = precision | |
| self._scale = scale |
Can we do this style instead to make it more explicit?
There was a problem hiding this comment.
this is a internal only feature, I think we don't want to expose this information to customer?
There was a problem hiding this comment.
OK, we can just keep it as kwargs then.
What do you think about moving the arguments onto _IntegralType instead of _NumericType? It looks like Snowflake only has a single precision for FLOAT and DECIMAL (docs), so we should never have to expose precision information for those types.
There was a problem hiding this comment.
+1 on using _IntegralType, I don't think we need _scale as it would always be 0
There was a problem hiding this comment.
i see, will fix this
|
I modified eq function to exclude _precision and _scale |
src/snowflake/snowpark/modin/plugin/_internal/snowpark_pandas_types.py
Outdated
Show resolved
Hide resolved
src/snowflake/snowpark/types.py
Outdated
| def __eq__(self, other): | ||
| def filtered(d: dict) -> dict: | ||
| return {k: v for k, v in d.items() if k not in ("_precision", "_scale")} | ||
|
|
||
| return isinstance(other, self.__class__) and filtered( | ||
| self.__dict__ | ||
| ) == filtered(other.__dict__) |
There was a problem hiding this comment.
why we overwrite the logic to exclude the new prop?
There was a problem hiding this comment.
here is a example test:
expected_fields = [
StructField("COL1", LongType(), nullable=True),
StructField("COL2", LongType(), nullable=True),
]
assert df.schema.fields == expected_fields
we exclude the private var so that these existing test does not fail
src/snowflake/snowpark/modin/plugin/_internal/snowpark_pandas_types.py
Outdated
Show resolved
Hide resolved
| ) == filtered(other.__dict__) | ||
|
|
||
| def __hash__(self): | ||
| return hash(repr(self)) |
There was a problem hiding this comment.
Why do we need a custom hash implementation for this class?
There was a problem hiding this comment.
TL:DR: this is to fix some test failure
when you defined eq, you need to redefine hash otherwise it is considered None
src/snowflake/snowpark/types.py
Outdated
| def filtered(d: dict) -> dict: | ||
| return {k: v for k, v in d.items() if k != "_precision"} | ||
|
|
||
| return isinstance(other, self.__class__) and filtered( | ||
| self.__dict__ | ||
| ) == filtered(other.__dict__) |
There was a problem hiding this comment.
I think whether excluding precision or not should be based upon the context._is_snowpark_connect_mode.
scos they might want to distinguish LongType by precision
|
|
||
| if kwargs != {}: | ||
| raise TypeError( | ||
| f"__init__() takes 0 argument but {len(kwargs.keys())} were given" |
There was a problem hiding this comment.
Grammar Error in Exception Message
The error message uses singular "argument" but will be grammatically incorrect when multiple arguments are passed.
# Current: "takes 0 argument but 2 were given" (incorrect grammar)Fix: Use proper singular/plural form:
arg_word = "argument" if len(kwargs) == 1 else "arguments"
raise TypeError(
f"__init__() takes 0 arguments but {len(kwargs)} {arg_word if len(kwargs) != 1 else 'was'} given"
)Or simply use plural consistently:
raise TypeError(
f"__init__() takes 0 arguments but {len(kwargs)} were given"
)Spotted by Graphite Agent
Is this helpful? React 👍 or 👎 to let us know.
|
I still kept the eq function in TimedeltaType class. |
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-2866776
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/SnowparkConnectRegressionRunner/220/
scos test